(19) 



J 



(12) 



Europffiscties Patentamt 
European Patent Office 
Office europ&n des brevets (11) EP 0 768 621 A2 

EUROPEAN PATENT APPLICATION 



(43) Date of publication: 

16.04.1997 Bulletin 1997/16 



(51) Int. CI. 6 : G06T5/20 



(21) Application number: 96202771.0 

(22) Date of filing: 04.10.1996 



(84) Designated Contracting States: 


• Gray, Robert Terry, 


DEFR GB 


c/o Eastman Kodak Co. 


Rochester, New York 14650-2201 (US) 


(30) Priority: 16.10.1995 US 6525 


• Matraszek, Tomasz Andrze], 


10.07.1996 US 675320 


c/o Eastman Kodak Co. 




Rochester, New York 14650-2201 (US) 


(71) Applicant EASTMAN KODAK COMPANY 




Rochester, New Yortc 14650-2201 (US) 


(74) Representative: Wagner, Karl H. f Dipl.-lng. etal 




GewQrzmflhlstrasse 5 


(72) Inventors: 


80538 MQnchen (DE) 


• Cok, David Roy, 




c/o Eastman Kodak Co. 




Rochester, New York 14650-2201 (US) 





CM 
< 

T— 

CM 
CO 

CO 
CO 



UJ 



(54) Method and apparatus for connecting pixel values in a digital image 



(57) A method and apparatus for editing continuous 
tone digital images representing pixels in which regions 
containing defects, called "defect regions", are unrro- 
ticeable or less visually discernible. Corrected pixel data 
for pixels in the defect region, called defect pixels, In the 
digital image are provided while data representing non- 
defect pixels outside of the defect region are not altered. 
These digital images may have one color channel (gray 
side) or multiple color channels. The apparatus and 
method includes a corrputer system which operates 
according to a set of instructions for providing the steps 
of: Selecting one of the defect pixels in the defect 
region, and then allocating a plurality of One segment 
through the selected defect pixel, wherein each line 
segment is composed of pixels in the image about the 
selected defect pixel. Next representative models are 
determined for the non-defect pixels of each fine seg- 
ment. A deviation is determined of the non-defect pixels 
along each line segment from their models. The line 
segment is selected having the lowest deviation, and 
then corrected values of the selected defect pixel are 
calculated based on the values of at least two non- 
defect pixels In the selected line segment on either side 
of the selected defect pixel. In another embodiment the 
corrected values of the selected defect pixel are based 
on intermediate values for the selected defect pixel 
which are calculated for each line segment and which 
contribute to the corrected values rotative to calculated 
weights based on the deviations for the line segments. 
Each pixel in the digital image is successively scanned, 
and the selecting of the defect pixels occurs while the 



image is being scanned. A corrected image is outputted 
containing the non-defect values in the digital image 
and the corrected pixel values for defect pixels in the 
defect region. 



RG. 3 



» AND 
STORE MAGt 



I LOCATE 0GFECT [f v 
I REOOHS ti MACE | 



BUKJD AMD STORE 
OEFBCT MAP 

I 



r 



SET VKHOOW OPERATOR / 
TO 1ST POP. 




115 



SELECT DEFECT PIXEL /" 



ADVANCE 
WINDOW 
OPERATOR TO 
KEXT PIXEL 



I 



ESTIMATE VALUES 
FOR PPffUn 

\ 



r 



STORE CSTUATED 
VAL UES FOR r ~ 
IN CORRECTED I 




§1 



OUTPUT ~J 
CORRECTED HIACC) 



Printed by Rank Hum (UK) Butfwttt fervbot 
2 U 2/1 4 



EP 0 768 621 A2 



Description 

This invention relates generally to improving the quality of digital images, and more particularly, relates to a method 
and apparatus for correcting long and narrow regions of defect pixels in a two<fimensional continuous tone digital 
s image. 

A digital image is a two-dimensional array of pixel elements. Examples of two-dimensional pixel arrays for digital 
images are 768 x 512 pixels or 1024 x 1024 pixete. Each pixel of a digital image may be represented by digital values 
which provide the intensity and color of the image. A digital image may be composed of a single color channel (gray 
scale) or multiple color channels, such as RGB or YCC, where each pixel in the digital image has a value for each color 
10 channel of an image. Generally, these digital images are called continuous tone digital images. 

Often, a digital image will have pixel regions with noisy or corrupted pixel values due to defects in the original digi- 
tized image or defects introduced into the image by the dig tizing process. Pixels within such regions are referred to as 
defect pixels and pixels outside those regions are referred to as non-defect pixels. 

Prior automatic image editing systems have had limited success in reconstructing these defect pixel regions. Typi- 
75 cally, these systems attempt to reconstruct the regions by estimating values for each defect pixel using values from the 
closest non-defect pixel or by averaging the values from a number of the closest non-defect pixels. Although this recon- 
struction is adequate in small compact defect regions of only a few pixels in size, it fails to provide proper reconstruction 
of larger (for exarrpie, long and narrow) defect regions. This results in reconstructed defect regions which are often vis- 
ually inconsistent with the non-defect pixels in the image. One reason for this inconsistency is that the selection of non- 
20 defect pixels used in estimating values of defect pixels may be improper, which causes blurring of the reconstructed 
image in the defect region. Another reason for this inconsistency may be the result of improper reconstruction of edges 
of objects in the image which intersect the defect regions. This results in reconstructed edges appearing bent or dis- 
continuous. 

A problem to be solved by the invention is to reconstruct defect regions which are visually consistent with the non- 
25 defect pixels in the image. 

An apparatus and method in accordance with the present invention includes a programmed computer system 
which operates on a set of instructions tor correcting defect pixels in a digital image, and which is coaled to an image 
reader, an image output device, and an user interface. The method set forth in the set of instructions includes the steps 
of: selecting one of the defect pixels in the defect region; allocating a plurality of line segments through the selected 
30 defect pixel, wherein each line segment is composed of pixels in the source image about the selected defect pixel; 
deterirfning at least one representative model of the non-defect pixel values in each line segment; determining the devi- 
ation of the non-defect pixel values in each line segment from their models; selecting the line segment having the lowest 
deviation; and determining corrected pixel values for the selected defect pixel based on the values of at least two non- 
defect pixels in the selected line segment 
35 An apparatus and method in accordance with another embodiment of the invention includes the steps of: selecting 
one of the defect pixels in the defect region; allocating a plurality of line segments through the selected defect pixel, 
wherein each line segment is composed of pixels in the source image about the selected defect pixel; determining at 
least one representative model of the non-defect pixel values in each line segment; determining the deviation of the 
non-defect pixel values in each fine segment from their model; determining first estimate pixel values for the selected 
40 defect pixel for each line segment based on the values of at least two non-defect pixels in each line segment; and deter- 
mining second values of the selected defect pixel based on the first pixel values, where the first pixel values contribute 
to the second pixel value relative to the deviations of each fine segment The second values of the selected defect pixel 
represent its corrected values. The above embodiments may also include the steps of scanning successively each pixel 
in the source image, selecting the defect pixel while the source image is being scanned, and generating a corrected 
46 image having the non-defect pixels of the source image, and the corrected values for each defect pixel. 

The advantage of the apparatus and method of the present invention is that the estimated defect pixel values in the 
reconstructed defect region of the source image are visually more consistent in color with the non-defect pixels in the 
image than in prior automatic image editing systems. This is particularly true for defect regions which are long and nar- 
row. This ultimately results in a corrected image having superior quality in the reconstructed defect regions, which are 
so either unnoticeable or visually less discernible to the human eye. 

Figure 1 is a block diagram of an apparatus for correcting values of defect pixels in a digital image in accordance 
with the present invention; 

Figure 2(a) is a pictorial view of a digital image with a defect pixel region^ 
55 Figure 2(b) is a pictorial view of a ctelert map for the digital irrage in Figure 2(a); 

Figure 2(c) is a pictorial view of the window operator used during correction of defect pixels in the digital image in 
Figure 2(a); 

Figure 2(d) is a pictorial view of a corrected digital image after correction of the defect pixels in the digital image in 
Figure 2(a); 
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Figure 3 is a flow chart illustrating a method for correcting defect pixels in a digital image in accordance with the 
present invention; 

Figure 4 is a flow chart illustrating a process for estimating corrected values for a defect pixel; 
Figure 5 is a flow chart of another embodiment illustrating a process for estimating corrected values for a defect 
s pixel; 

Figure 6 is a flow chart illustrating an alternative embodiment for defining the line segments in the process for esti- 
mating corrected values for a defect pixel in Figures 4 and 5; 

Rgure 7 is a pictorial view of the digital image of Figure 2(a) illustrating a line segment being defined as described 
in Figure 6; and 

10 Rgure 8 is a pictorial view of an example illustrating line segments defined about a defect pixel. 

An apparatus 8 in accordance with the present invention includes a programmed computer system 10 coupled to 
an image reader 20. an image output device 24, and a user interface 22. Computer system 10 operates according to a 
set of instructions to correct cfig'rtal images by correcting values of defect pixels within a defect region in the digital 

is images. With the apparatus and method, defect regions in digital images can be corrected so that regions are entirely 
unnoticeable to a viewer or less visually discernible because values for defect pixels have been estimated which are 
consistent with the overall image. 

Referring more specifically to Rgure 1 . computer system 10 includes a processor 1 2, a memory 14 and a hard disk 
18, which are coupled tog ether with an internal bus 16 for communicating both data and addresses between these com- 

20 ponents. Preferably, digital images are stored on hard disk 18 in system 10. 

Image reader 20 is coupled to computer system 10, and provides a digital image to the computer system 10 via 
internal bus 16. This digital image is stored in memory 14. In this particular embodiment, image reader 20 is a digital 
scanner or digital camera, such as Kodak Professional Photo-CD 4045 Film Scanner, although any type of device for 
providing a digital image may be used. Computer system 10 may also receive digital images from other sources, such 

25 as digital images stored in hard disk 18. Each digital image is composed of an array of pixel values having one or more 
color channels. 

Image output device 24 is coupled to computer system 10. In this particular embodiment, output device 24 may be 
any type of digital image output device, such as a video display, a printer, an external non-volatile memory storage 
device, or a transmission line to other computers within a network. Images stored in memory 14 may be output to image 
so output device 24 through internal bus 16. 

User interface 22, such as a keyboard or mouse device, is coupled to computer system 10 and allows for a user to 
control and interact with apparatus 8, such as building of a defect map of the digital image stored in memory 14. User 
inputs through user interface 22 are descrfced later. 

Referring to Figures 2(a4) and 3, the operation of apparatus 8 will be discussed. When computer system 10 is 
35 started at step 98, system 10 receives a digital image from image reader 20 and stores the image in memory 14 (step 
100). Alternatively, this image may have already been stored in memory 14. The image stored in memory 14 will here- 
inafter be referred to as the "source image." An example of the source image 25 is shown in Rgure 2(a) with defect 
region 26. The source image may be comprised of a single color channel (monochrome), or have multiple color chan- 
nels, such as RGB. 

40 Once the source image is stored in memory 14, then in step 102 the defect regions in the source image are located. * * 
Preferably, this is performed by the user identifying the approximate image areas containing the defect regions through 
user interface 22. and then processor 12 automatically detecting the defect pixels in those areas identified by the user. 
One system for identifying defect pixels is disclosed in U.S. Patent Application Serial No. 08/057,942 f led May 4, 1993, 
now abandoned in fervor of Continuation bearing Serial No. 08/412,351 in the name of Robert T. Gray and assigned to 

45 the same assignee as this application, and is herein incorporated by reference. Briefly, this defect pixel identification 
system automatically identffies email local regions of an image which are anomalous in both brightness/oofor, local con- 
trast, and size, and outputs a map of the defect pixels in the image. Alternatively, the defect pixels may be selected man- 
ually by the user, which can be performed by a user generating a digital mask via user interface 22 (for example, by a 
graphic "painting" software application) identifying the defect pixels. 

so Once the defect regions are located, a defect map is built and stored in memory 14 (step 104). In the defect map, 
pixels with a specified value correspond to defect pixels in the source image. One example of a defect map is shown in 
Figure 2(b) for the defect region 26 of the source image 25 in Figure 2(a). In this example, defect pixels are represented 
by bits set to zero (black), and non-defect pixels are represented by bits set to one (white). 

Next a window operator is set, preferably centered, on a first pixel injhe source image (step 106). An example of 

55 the window operator 27 in operation is 6hown in Rgure 2(c). In computer system 10, the window operator is used in 
performing correction of defect pixel values in the source image. The window operator defines a region of interest in the 
source image about a pixel centered within the region. In the preferred embodiment the shape and size of this window 
operator is a square region, where each side or boundary of the square region is X in size. However in another embod- 
iment the window operator may be a circular region having a diameter X. This window operetta is used by processor 
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1 2 to scan across each pixel in the lines of the source image. The scanning pattern of the window operator is not critical 
as long as all defect pixels are visited during scanning. For example, a raster-scan pattern may be used. 

In step 1 07, a check is made to determine whether this pixel set within the window operator is a defect pixel by ref- 
erencing the defect map stored in memory 14. If the pixel is a non^defect pixel, then in step 109 the values of this non- 

s defect pixel are stored in an area of memory 14 which is allocated for storing the corrected source image ("corrected 
image"). If the pixel is a defect pixel, then the defect pixel is selected ("PIXELsa.") for subsequent correction (step 108). 
Next, a corrected value in each channel of PIXEL^ is estimated (step 110). Figure 2(c) shows an fl lustration of window 
operator 27 and the estimation of corrected values for a defect pixel in the defect region 26 of the source image 25. The 
process for estimating corrected values is described in greater detail in Figure 4. 

10 Referring to Figure 4 ( the process for estimating corrected values for PIXELq^ is started at step 198. The Gne seg- 
ment size is set to X for every direction (step 200). Preferably, X Is at least twice the maximum width of the defect region 
in the source image. A SET of line segments or vectors is defined or allocated through PIXELs EL wherein each line seg- 
ment is composed of both defect and non-defect pixels in the source image on one or both sides of PIXELs EL (step 
202). In this embodiment the total number of pixels defect and non<Jefect pixels, in each line segment is maintained 

is the same, which corresponds to the square window operator. Alternatively, each fine segment can have the same length 
X defined as the Euclidean distance, which corresponds to the circular window operator. However, in other embodi- 
ments the total number of pixels in each line segment may vary depending on the defect pixel and line segment, as will 
be descrbed later. 

SET is composed of N number of line segments, where each fine segment is represented by the following notation: 

20 In SET. n refers to each line segment (n=1 to N). Each line segment n has pixels having values for K number of color 
channels. The nth line segment is referred to as LS,,. The kth channel of the nth line segment in SET is referred to as 
L$ nK For example, where SET has 4 line segments (N=4), and each pixel has 3 channel values (K=3), the 2nd channel 
of the 3rd line segment in SET is Uh?. 

Preferably, the line segments are each at equal radial angles at 45* or 22.5° intervals with respect to each other 

85 about PIXELs EL , although the angles can vary as needed. The number of fine segments in SET depends on size of this 
angle. For example, radial angles at 45° intervals provide 4 fine segments in SET, while radial angles at 22.5° intervals 
provides 8 fine segments in SET. Preferably, each fine segment is composed of pixels which are less than 0.5 pixel dis- 
tance away from a fine through PDCEUsq. at the radial angle associated with each respective line segment 

Referring to Figure 8, an example of a SET of 4 line segments (H, V, D1 , and D2) is shown at 45° spacing to each 

30 other, where each fine segment is composed of a total of 22 pixels and has 11 pixels on each side of PDCELsa,. 

Referring back to Figure 4, once the SET of line segments is defined . then at step 204 any fine segments which do 
not contain a minimum threshold number of non-defect pixels on each side of PDCELsq, are considered not vaBd line 
segments, and are removed from SET Preferably, the minimum threshold is set to one-half the number of pixels on one 
side of a line segment For exarrple. if each line segment has 8 pixels, then 4 pixels are on each side of PIXELs EL and 

35 the minimum threshold is 2 pixels. Accordingly, any fine segment which does not have at least 2 non-defect pixels on 
each side of PIXEL^ is removed from SET. 

Next a check is then performed to determine whether the remaining number of line segments N in SET is greater 
than zero (step 206). If N is not greater than zero, then the "no" branch is taken to step 207 where all the line segments 
previously removed are replaced in SET. Once the fine segments are replaced, a query is made to determine whether 

40 any of the line segments have non-defect pixels (step 209). If there are no non-defect pixels in the line segments of SET, 
then the 'no" branch is taken to step 21 1 where the line segment size X is increased by Y This accordingly expands the 
window operator region by Y, thereby increasing the possbiltty that when SET is redefined line segments will include 
non<lefect pixels. A new SET of line segments is redefined through PIXEL^el at this increased line segment size at 
step 202, and the process as described above is repeated. H the line segments at step 209 contain any non-defect pix- 

45 els, then the "yes" branch is taken to step 213 where an estimated value for each channel k of PfXELsa. is calculated 
based on the weighted average of the non-defect pixels in SET. These weights are determined by their respective dis- 
tances from PIXELsel- 

The calculation for estimating values for each channel k of PIXEL^ at step 213 preferably Is as follows: 

50 Q 

PIXEL SELk = tig (D 



where: 

I b index for each non-defect pixel in SET; 
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q = total number of non-defect pixels in SET; 

P = the value of the k* channel of the i* non-defect pixel in SET; and 

W, b the weight for the i m non-defect pixel in SET, where 



10 X| and yi = the x and y coordinates of the I th non-defect pixel in SET in the source image, and 

x SEL and y 3EL = the x and y coordinates of PIXEL^ EL in the source image 

Once the estimation of values for PDCELq EL is completed (step 218) processing then goes to step 1 12 in Figure 3. 
If at step 206 the number of line segments N is greater than zero, then the "yes* branch is taken to step 208 where 

is a functional model is calculated for the non-defect pixel values for each channel k of each line segment in SET. The 
functional model calculation is performed by fitting the values of the non-defect pixels in a line segment to a mathemat- 
ical model, such as a flat field fit. linear fit or quadratic fit. This fitting is based on the values of the non-defect pixels, 
and the distances between them. A variety of techniques may be used for modeling the non-defect pixel values, and the 
particular technique chosen will affect model accuracy, and require a different minimum number of non-defect pixels. 

20 One exairple of th's technique is shown in W. PRESS. S. TEUKQLSKY, W. VETTERLING, AND B. FLANNERY. 
NUMERICAL RECIPES IN FORTRAN, Ch. 15 (2d ed.. 1992). which is herein incorporated by reference. In the pre- 
ferred errtxxliment the functional model calculation at step 208 uses a linear least-squares fit. which requires at least 
one non-defect pixel lying on each side of PIXELs EL . Earlier performed step 204 may be used to assure that this mini- 
mum is met for all line segments in SET. 

25 Next at 6tep 2 1 0, a mean square fitting enror (MSE) is calculated of the non-defect pixel values from the functional 
model for each channel of each line segment in SET. The MSE calculation is a statistical measure of the deviation of 
the non-defect pixel values in a line segment from an estimate of their values based on their functional fit. Thus, the 
lower a MSE value the more consistent to the model are the non-defect pixel values. For a given channel and line seg- 
ment. MSE is calculated by determining the difference between the value of each non-defect pixel from its value esti- 

30 mated along the functional model. The resulting difference tor each non-defect pixel is then squared, and the sum of 
these squares is averaged to obtain the MSE. (In the case of a fiat field fit for the functional model caicuiation. the MSE 
is equivalent to the variance of the rwwletect pixels in the line segment) Mean square fitting error of the k? 1 channel of 
the n* fine segment (MSE n is calculated by: 



40 



where: 

M n - number of non-defect pixels in LS n ; 

I m index for each non-defect pixel in LS n from 1 to M n ; 

45 p, Ak « the actual value of the i* non-defect pixel in channel k in LS n ; and 

Pi,n,k " the to 1 *** 0 " 81 wo6e{ estimate value of the i* non-defect pixel in channel k in LS„. 

Then at step 212, a calculation is performed for each line segment wherein each channel of the line segment is 
combined to provide a total mean square fitting error value 



so 



(MSE Totaln ) , 



where: 

55 



MSE^olXMSE^ (4) 
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K = the total number of channels (for RGB, K=3); and 
k = index for each channel from 1 to K. 

The line segment having the lowest MSE Tota) is then selected (step 214). The selected line segment represents the 

5 preferred line segment tor determining estimated values for PIXELsel- A" estimated value for each channel k of PIX- 
ELsa. is then interpolated using at least one non-defect pixel of the selected line segment on each side of PlXELs EL 
(step 216). In this manner, by selecting the line segment with the lowest total mean square fitting error, the line segment 
representing non-defect pixels which are most consistent to their functional model across all channels is used In deter- 
mining the estimated values for PlXELg a . Once the estimation of values for PIXEL^ is completed (step 218) then 

10 processing goes to step 1 12 in Figure 3. 

Referring to Figure 8, an example of the processing in Figure 4 is described below. Figure 8 shows an expanded 
view of a source image with a defect region extending from Region 1 to Region 2, wherein each region is composed of 
pixels with similar pixel values, which are distinct between regions. There are 4 tine segments in SET labeled H, V, D1 
and D2, which are defined through PIXEL^ at the center or intersection of the line segments (step 202). Once 

15 defined, line segment H and D2 are removed from SET because each has a side without an/ non-defect pixels, and 
therefore is below a rnininrum threshold of 4 non-defect pixels (step 204). Next a functional model is fitted and a mean 
square fitting error is calculated for remaining line segments V and D1 (steps 208-210). In the angle color channel 
(K=1) example of Figure 8. the mean square fitting error for line segment D1 is lower than the mean square fitting error 
for line segment V. This is due to the excessive fitting error for dissimilar non-defect pixel values along line segment V 

20 extending through Region 1 and Region 2. In contrast, the non-defect pixels of fine segment D1 all lie within Region 1 
and have uniform pixel values. Thus line segment D1 has a lower fitting error than line segment V. Consequently, at step 
214 line segment D1 is selected, and at step 216 PIXELsa. is replaced with an estimate value based upon the non- 
defect pixels in line segment D1 . 

Referring to Figure 5, an alternative embodiment for estimating corrected values for PIXELsel e illustrated. Conre- 

25 sporting steps in Figure 5 have numeral designations which correspond to those in Figure 4 and will not be described 
again. Accordingly, once the mean square error calculations are made at step 210, then a weight (w n J is calculated at 
step 220 for each channel kfor each line segment n in SET, where: 



30 



MSE 



(5) 



LMSE,* 
i-t 



35 As shown by the equation (5), the lower the mean square fitting error for a given line segment in a given channel, the 
greater will be the calculated weight. Next, an trrter mecfiate estimate value of PIXELsq. (IGI^r) is interpolated for each 
channel k for each line segment in SET (step 222). Once the interpolated process is completed, a final estimate value 

of PIXELgEL 

<PGL SELk ) 

is calculated for each channel k (step 224), where: 

45 N 

£w lik IGL i(k 

so U1 

Tlius, the final estimate value of PIXEL^ is based upon a contribution of each of the calculated intermediate estimate 
values for each channel of each fine segment relative to their respective weights. To illustrate this, if for example the line 
segments are those shown in Figure 8, then intermediate estimate values for PIXELsa. are calculated for line segments 
65 V and D1 because aD other line segments were removed at step 204. But since tine segment D1 has a lower mean 
square fitting error than line segment V, the intennedate estimate value for line segment D1 contributes significantly 
more to the final estimate value of PIXELsa. ftan the intermediate estimate value for line segment V. Once the estima- 
tion of values for PIXELsa. » completed (step 216) then processing goes to step 1 12 in Figure 3. 

Referring back to Figure 3, once estimated values for PIXELsa. are calculated, then at step 1 12 they are stored in 
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the corrected image area of memory 14. A check is then made at step 1 14 to determine whether the end of the source 
image has been reached. In other words, have all pixels in the source image been scanned by the window operator. If 
not then the "no" branch is taken to step 1 15 where the window operator is advanced, and set preferably centered, to 
the next pixel in the image and steps 107-1 14 are repeated as previously described. 
5 Once the end of image is reached, the "yes" branch is taken from step 1 14 to step 116, where the corrected image 
stored in memory 14 is output to the image output device 24 at step 116 and the end of the process at step 118. An 
example of the corrected image is shown in Figure 2(d), wherein the defect 26 has been corrected and is no longer vis- 
ually discernible. 

In two alternative embodiments, the size of each line segment at step 202 in Figures 4 and 5 is variable along its 

10 respective radial line through PIXELs EL . Thus, step 200 for setting the line segment size is no longer needed. Line seg- 
ments are extended on both sides of PIXELs EL . The composition of each line segment is determined by extending each 
line segment along a vector (that is. its respective radial line) from each side of PIXEL^a. as described below. 

In the first alternative embodiment, line segments are separately extended on each opposite side of PIXEL©^ on 
a vector along its respective radial line until each side is coirposed of a certain number of non-defect pixels. ND, (not 

is shown or described with respect to Figures 4 and 5). Preferably, ND represents two or three non-defect pixels. If a line 
segment cannot be extended on either side of PIXELs^ to include ND non-defect pixels, then that line segment is not 
included in SET. This can occur because the line segment extension reached the outer edge of the source image (or 
defect map) or a maximum distance from PfXEUsa. without locating ND non-defect pixels. Once both skies of any line 
segment include ND non-defect pixels, it is assured that the line segment has a proper number of non-defect pixels for 

20 subsequent processing. 

Referring to Figures 6 and 7. the second alternative embodiment will new be discussed. Figure 6 is a flow chart 
descrfoing the extension of one side of line segment LS„ from PDCEUs EL . Starting at step 226. multiplier A is first set to 
1 (step 228). Variable Q is then set to a desired initial number of non-defect pixels from PIXELs EL , and variable E to a 
desired increment number of non-defect pixels (step 230). Next, the first Q adjacent non-defect pixels for the line seg- 

25 ment LS„ extension from PIXELsa_ is set as Group 1 along a vector corresponding to yne segment LS n radial line (step 
232). A series of stopping conditions for extending fine segment LSn then checked. Fast, if Group 1 cannot be 
established either due to the lack of Q adjacent non-defect pixels or that the edge of the source image (or defect map) 
was reached, then path 262 from step 232 is followed to step 260 where line segment LS n is net included in SET. Next 
a check is performed to determine whether the number of pixels in Group 1 exceeds a maximum threshold of line seg- 

so ment non-defect pixels THmax (step 236). If so, then the "yes* branch is taken to step 257 where the line segment LS n 
extension stops at the pixel in Group 1 before THmax was exceeded. TH^^x represents the maximum number of non- 
defect pixels any one side of PIXELs EL may be composed of. 

If the number of pixels in Group 1 does not exceed TH^^x, then the 'no" branch is taken to step 238 where the first 
Q + (A*E) adjacent non-defect pixels from PIXEL^ along the same vector as Group 1 is set as Group Z rf Group 2 

55 cannot be est&ftshed either due to a lack of Q + (A*E) adjacent non-defect pixels, or that the edge of the source image 
(or defect map) was reached, then path 263 is taken to step 261 where line segment LS„ extension stops at the last 
pixel in Group 1 . Q represents a base number of non-defect pixels in line segment LS n , and E represents the number 
of successive non-defect pixels added to that base number in each iteration of the extension process, as will be shown 
below. 

40 Figure 7 illustrates an example of extending line segment LS n along a vector 26 from PIXEL SEL in defect region 26 
of source image 25. The black dots along vector 28 in Figure 7 represent adjacent non-defect pixels from PDCEL^a. (not 
drawn to scale). In this example, Q equals 3 and E equals 2. Thus, Group 1 comprises the first three adjacent non- 
defect pixels closest the PIXELsh.. and Group 2 comprises five non-defect pixels, that is, the three non-defect pixels in 
Group 1 and the next two adjacent non-defect pixels from PIXEL^ after the non-defect pixels cf Group 1 . 

45 Next a functional model is calculated for each channel k of the pixels in each group. Then, at step 240 a mean 
square fitting error (MSE g £ is calculated for each channel k of their non-defect pixels from their model, where g - 1 and 
2, representing Group 1 and Group 2 respectively. Functional model and MSE calculations were described earlier in 
reference to Figure 4. Next, the MSE's for each channel of each group are summed to provide a sum mean square error 
value (SMSEg) for each group g (step 242), where: 

50 

K 

SMSE g =£MSE gR (7) 
k-1 

55 

In the preferred embodiment SMSEg is calculated only when necessary that is the SMSEg is stored as SMSE 1 for the 
next extension step. 

Next a RATIO is calculated for the sum mean square errors of Group 2 to Group 1 , RATIO « SMSEg/Max [SMSE t . 
TVy. where TH 2 is greater than 0 and the Max function selects the greater of the two terms SMSE 1f and THz (step 
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244). 

The next three checks are based upon SMSE 1t SMSE 2 and RATIO. The first check is performed only once at the 
beginning of the extension process when A « 1 . The first check is whether the sum mean square error calculated for 
Group 1 , SMSE, , is greater than a first threshold. TH, (step 246). If so, then the "yes" branch is taken to step 254 where 
line segment LS„ is not included in SET. The second check is whether the sum mean square error of Group 2, SMSE 2 , 
is greater than the first threshold, TH 1 (step 247). The third check is whether RATIO is greater than a third threshold, 
TH 3 (step 248). These three thresholds TH t , Thfe, and TH 3 are empirically determined such that the more statistically 
consistent the values of the non-defect pixels are within each group to their model, the further line segment LS„ will 
extend from PIXEL^. Preferably, these thresholds are selected to provide that TH 1 is greater than TH 2 , and TH 3 is 
greater than 1 . If all three checks at steps 246. 247, and 248 are false, then their "no" branches are taken to step 250 
where the Group 1 pixels are replaced with the pixels of Group 2. Then, A is indexed by one (step 251 ) and steps 236- 
248 are repeated as previously described, wherein Group 2 now includes the next E adjacent non-defect pixels from 
PIXELq EL in addition to the pixels of Group 1 . Using Figure 7 as an example. Group 1 now has five non-defect pixels 
from PDCELq^ and Group 2 has seven non-defect pixels from PIXEL^ along vector 28. If either the second check 
(step 247) or third check (step 248) is true, then their "yes" branch is taken to step 253 where the line segment LS„ 
extension stops at the last non-defect pixel furthest from PIXELea. in Group 1. Groups 1 and 2 will continue to be 
expanded in the above manner until at least one stopping condition is satisfied. 

Once the extension of one side of line segment LS„ from PIXELsa. is completed, without an indication of non-indu- 
sion in SET, the above process is repeated for the other side of fine segment LS n . In this manner, the total composition 
of defect and non-defect pixels for each side of line segment LSn * determined or fixed. The above is repeated for each 
line segment through PIXEL^. Finally, the SET of fine segments defined at step 202 comprises the line segments 
which are extended on both sides of PIXELsel without a non-inclusion indication. 

Of the above two alternative embodiment, the second embodiment is preferred because it employs statistical anal- 
ysis of non-defect pixel values (for exanrpte, SMSE) in extending line segments, rather than a constant number of non- 
defect pixels ND. Further, in the second embodiment those skilled in the art can appreciate other types of stopping con- 
ditions may also be used than those described to provide different line segment extension criteria. 

Preferably in this apparatus, once non-defect pixels have been estimated and stored in the corrected image (steps 
110 and 112 in Figure 3), their estimated corrected values are not used In subsequent correction of other non-defect 
pixels In the source image. However, where after a single scan of the source image defect pixels remain uncorrected, 
corrected pixel values alternatively can be used in a second scan for correcting defect pixels. This may be required 
where the defect region is very large, or the defect region lies near the boundaries of the source image. 

From the foregoing description, it will be apparent that there has been provided an improved method and apparatus 
for correcting pixel values in a digital image, variations and modifications in the herein described system in accordance 
with the invention will undoubtedly suggest themselves to those skilled in the art. Accordingly, the foregoing description 
should be taken as illustrative and not in a limiting sense. 
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1 . A method for correcting one or more defect pixels in a source image with non-defect pixels and the defect pixels, 
each of the defect pixels and the ncxvdefect pixels being represented by at least one defect pixel signal and non- 
defect pixel signal, respectively, the method comprising the steps of: 

selecting one of the defect pixels; 
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allocating a plurality of line segments through the selected defect pixel, wherein each the line segment is com- 
posed of the defect pixels and the non-defect pixels about the selected defect pixel; 
determining at least one representative model of the non-defect pixel signals along each the line segment; 
determining the deviation of the non-defect pixel signals along each the line segment from the model corre- 
s sponding to the line segment; 

selecting the line segment having the lowest deviation; and 

determining corrected pixel signals for the selected defect pixel based on at least two of the non-defect pixel 
signals in the selected tine segment. 

10 2. A method as claimed in Claim 1 wherein one or more of the defect pixels are located within a defect region of the 
source image, and the step of 

selecting one of the defect pixels includes selecting one of the defect pixels in the defect region. 

is 3. A method as claimed in Claim 1 wherein: 

the defect pixel signals represent channel signals of their corresponding defect pixels; 
the non-defect pixel signals represent channel signals of their corresponding non-defect pixels; 
the model determining step further comprises determining a representative model based upon correspond* 
ing channel signals of the non-defect pixels along each the fine segment; and 
20 the deviation determining step further comprises determining a deviation of corresponding channel signals 

of the non-defect pixels along each the line segment from the model of the channel signals of the non-defect pixels 
along the line segment 

4. A method as claimed in Claim 3 wherein: 

25 the deviation determining step further comprises the step of determining a total for each the line segment 

based on the deviation of the channel signals of the non-defect pixels; 

the line segment selecting step further comprises the step of selecting the fine segment having the lowest 
total; and 

the pixel signals determining step further comprises determining corrected channel 6ignal6 for the selected 
so defect pixel based on the channel signals of at least two of the non-defect pixels in the selected One segment. 

5. A method for correcting one or more defect pixels in a source image with non-defect pixels and the defect pixels, 
each of the defect pixels and the non-defect pixels being represented by at least one defect pixel signal and non- 
defect pixel signal, respectively, the method comprising the steps of: 

55 

selecting one of the defect pixels; 

allocating a plurality of line segments through the selected defect pixel, wherein each the line segment is com- 
posed of the defect pixels and the non-defect pixels about the selected defect pixel; 
determining at least one representative model of the non-defect pixel signals along each the line segment; 
40 determining the deviation of the non-defect pixel signals along each the line segment from foe model corre- 

sponding to the line segment; 

determining first pixel signals representing the selected defect pixel for each the line segment based on at least 
two non<Jefect pixel signals in the line segment; and 

determining second pixel signals representing the selected defect pixel based on the first pixel signals for each 
45 line segment wherein the second pixel signals are based upon a contribution of each the first pixel signals of 

each the line segment relative to their respective deviation. 

6. An apparatus for correcting values of defect pixels in a source image having a programmed computer with memory 
tor storing the source image, the stored source image comprising a plurality of pixels having signals representing 

so values of one or more color channels, and a defect region comprising the pixels within the plurality having corrupted 
values, wherein the pixels in the defect region are defect pixels, and the pixels outside the defect region are non- 
defect pixels, the apparatus comprising: 

means for selecting one of the defect pixels in the defect region; y 
55 means for allocating a plurality of line segments through the selected defect pixel, wherein each the line seg- 

ment is composed of pixels in the source image about the selected defect pixel; 

means for determining for each channel of each the line segment a representative model of the non-defect pix- 
els values; 

means for determining for each channel of each the Gne segment the deviation of the non-defect pixels values 
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in the line segment from their representative model; 

means for determining a total for each line segment based on the deviation for each channel of the line seg- 
ment; 

means for selecting the line segment having the lowest the total deviation from among the plurality of fine seg- 
ments; and 

means for determining a value for each channel of the selected defect pixel based on the values of at least two 
of the non-defect pixels in the selected line segment. 

An apparatus for correcting values of defect pixels in a source image having a programmed computer with memory 
for storing the source image, the stored source image comprising a plurality of pixels having signals representing 
values of one or more color channels, and a defect region comprising the pixels within the plurality having corrupted 
values, wherein the pixels fn the defect region are defect pixels, and the pixels outside the defect region are non- 
defect pixels, the apparatus comprising: 

means for selecting one of the defect pixels in the defect region; 

means for allocating a plurality of line segments through the selected defect pixel, wherein each the line seg- 
ment is composed of pixels in the source image about the selected defect pixel; 

means for determining for each channel of each the One segment a representative model of the non-defect pix- 
els values; 

means for determining for each channel of each the line segment the deviation of the non-defect pixels values 
in the One segment from their representative model; 

means for determining for each channel of each the line segment a first value for the selected defect pixel 
based on the values of at least two non-defect pixels in the line segments; and 

means for determining a second value for each channel of the selected defect pixel based on the first value for 
each channel of each line segment, wherein the second value is based tpon a contribution of each the first 
value for each channel of each the line segment relative to their respective the deviation. 

An apparatus for correcting one or more defect pixels in a source image with non-defect pixels and the defect pixels, 
each of the defect pixels and the non-defect pixels being represented by at least one defect pixel signal and non- 
defect pixel signal, respectively, the apparatus comprising: 

means for selecting one of the defect pixels; 

means for allocating a plurality of line segments through the selected defect pixel, wherein each the line seg- 
ment is composed of the defect pixels and the non-defect pixels about the selected defect pixel; 
means for determining at least one representative model of the non-defect pixel signals along each the line 
segment; 

means for determining the deviation of the non-defect pixel signals along each the line segment from the model . 
corresponding to the line segment; 

means for selecting the line segment having the lowest deviation; and 

means for determining corrected pixel signals for the selected defect pixel based on at least two of the non- 
defect pixel signals in the selected line segment 

An apparatus for correcting one or more defect pixels in a source image with non-defect pixels and the defect pixels, 
each of the defect pixels and the non-defect pixels being represented by at least one defect pixel signal and non- 
defect pixel signal, respectively, the apparatus comprising: 

means for selecting one of the defect pixels; 

means for allocating a plurality of line segments through the selected defect pixel, wherein each the line seg- 
ment is composed of the defect pixels and the non-defect pixels about the selected defect pixel; 
means tor determining at least one representative model of the non-defect pixel signals along each the line 
segment; 

means for determining the deviation of the non-defect pixel signals along each the line segment from the model 
corresponding to the line segment; 

means for determining first pixel signals representing the selected defect pixel for each the One segment based 
on at least two non-defect pixel signals in the line segment; and 

means for determining second pixel signals representing the selected defect pixel based on the first pixel sig- 
nals for each line segment wherein the second pixel signals are based tpon a contrfoution of each the first 
pixel signals of each the line segment relative to their respective deviation. 
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1 0. An apparatus for correcting values of defect pixels in a source image having a programmed computer with memory 
lor storing the source image, the stored source image comprising a plurality of pixels having signals representing 
values of one or more color channels, and a defect region conprising the pixels within the plurality having corrupted 
values, wherein the pixels in the defect region are defect pixels, and the pixels outside the defect region are non- 
defect pixels, the apparatus'comprising: 

the processor selects of one of the defect pixels in the defect region; allocates a plurality of line segments 
through the selected defect pixel wherein each the line segment is composed of pixels in the source image 
about the selected defect pixel, determines for each channel of each the line segment a representative model 
of the non-defect pixels values, determines for each channel of each the line segment the deviation of the non- 
defect pixels values in the line segment from their representative model/determines a total for each line seg- 
ment based on the deviation for each channel of each line segment, selects the line segment having the lowest 
the total from among the plurality of tine segments, and determines a value for each channel of the selected 
defect pixel based on the values of at least two of the non-defect pixels in the selected line segment. 
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